Add ST Grammar#1
Draft
simoes-tiobe wants to merge 7 commits into
Draft
Conversation
Several places where the IEC grammar reduces a bare identifier through multiple aliased or overlapping rules (type vs. function-block names, derived type-name variants, access-path segments) are genuinely ambiguous from syntax alone - which one applies is only decidable with a symbol table - so those spots are collapsed onto the existing shared `type_name`/`symbolic_variable` rules instead, deferring the distinction to a later semantic pass. Other conflicts are fixed more locally: - signed_integer/real_literal fold their optional sign into the token regex so tree-sitter's longest-match lexing picks them over a bare unary-minus operator. - bit_string_literal requires its type prefix, since an unprefixed integer was ambiguous with a plain integer literal. - character_string (used where no STRING/WSTRING keyword disambiguates) is now a single unaliased rule instead of a single-vs-double-byte choice that shared the same delimiter and representation set. - param_assignment's `NOT var_name => var` vs. unary_operator's `NOT expr` is a genuine local ambiguity (resolvable with more lookahead, not by restructuring), so it's handed to GLR via `%conflicts`. Also drops fb_name_decl, made dead by the type_name consolidation.
Five GLR conflicts remained after the previous pass, all the same root cause: a bare identifier reachable through more than one differently labeled rule with nothing in the following tokens to tell them apart (program_access_decl/access_declaration's type name, array_specification's element type, data_source's resource/program/global-var prefix, global_var_reference's own prefix/suffix, and _instance_specific_init's fb_name vs variable_name). Each is resolved the same way the file already resolves this pattern elsewhere: drop the redundant labeled alternative and rely on the single already-reachable rule (type_name, symbolic_variable), letting a symbol table (not this CFG) resolve which kind it actually is. Dead rules left over from these fixes (non_generic_type_name and its six-way identifier aliasing, program_output_reference, function_block_type_name's derived-name half) are removed, and standard_function_block_name's FIXME placeholder is filled in with the fixed set of names from IEC 61131-3:2003 2.5.2.3 (SR/RS, R_TRIG/F_TRIG, CTU/CTD/CTUD variants, TP/TON/TOF), with a new call site for it. Also adds two pieces of parser infrastructure that were missing entirely: `%extras` for `(* ... *)` and `//` comments (real ST code is full of them), and a fix for the generated parser's start rule, which had no explicit axiom and so defaulted to whichever rule happened to land first after %include flattening - silently `elementary_type_name` rather than `library_element_declaration`.
Replaces tests/literals.txt, which had gone stale (written against an earlier structure of the grammar and no longer matching current rule names/shapes) and had never actually been exercised, since ts-bnf-tool generation was failing on the conflicts fixed in the previous commit. tests/community_samples.txt has six small, complete, hand-written ST declarations pulled from real MIT-licensed open-source projects (not AI-generated) - WengerAG/structured-text-utilities and tkucic/UniTest - covering a spread of rules (enum/struct TYPE, functions with nested calls, VAR_TEMP, IF/ELSE, a function block with FOR/array/struct-field access). tests/SOURCES.md has the license/attribution details. Only one of the six currently parses clean; the rest hit real grammar gaps the corpus surfaced (mandatory `;` after already-block-terminated constructs, enumerated_value's mandatory qualifier, STRING[n] as a general type name, FUNCTION's single non-repeatable, VAR_TEMP-less declarations slot). Documented in tests/FINDINGS.md with examples, root causes, and recommended fixes, held for review before changing the grammar further.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Add ST Grammar